Description:

PayU is hiring a Senior Data Engineer for its dynamic global data infrastructure group. Positioned at the critical junction of big data architecture and machine learning operations (MLOps), this role is responsible for driving the backbone platforms that process massive transactional datasets in high-growth markets. Moving beyond basic report generation, you will build sophisticated, real-time feature engineering pipelines and scalable ML model infrastructure. The day-to-day scope balances optimization of cloud data warehouses, constructing automated ETL/ELT pipelines, implementing low-latency CDC (Change Data Capture) systems, and managing robust feature store architectures that enable data scientists to seamlessly deploy and scale production-grade predictive models.

Key Technologies:

Python, SQL, NoSQL, AWS (EMR, S3, Redshift, MSK), Apache Spark (PySpark), Apache Airflow, Debezium, Apache Hadoop, Feature Stores

Requirements:

• 3–6 years of dedicated professional experience operating within high-volume data engineering setups, preferably within fast-paced startup or agile global fintech frameworks.
• Strong expertise in distributed computing architectures and exceptional logical problem-solving skills to manage terabyte-scale data pipelines.
• In-depth development experience utilizing public cloud infrastructure services, specifically Amazon Web Services (AWS) environments including EMR for cluster computing, S3 data lakes, Redshift warehousing nodes, and Managed Streaming for Apache Kafka (MSK).
• High technical command over standard data platforms: processing via Apache Spark (PySpark), scheduling via Apache Airflow, data replication via Debezium for Change Data Capture (CDC), and storage systems within Apache Hadoop.
• Advanced programming skills in Python and SQL, paired with a deep understanding of relational data tuning, indexing configurations, architectural schema enhancements, and managing NoSQL databases.
• Proven hands-on footprint designing analytical data models, structuring physical schemas, and developing clean DataMarts tailored for executive business intelligence and automated reporting.
• Clear structural understanding of machine learning model deployment, automated retraining workflows, and production maintenance patterns leveraging Feature Store architectures and engineering best practices.
• Ability to collaborate closely with business stakeholders, parse raw data warehouse demands, and transform functional user goals directly into robust, scalable software frameworks.
• Background exposure or structural experience setting up, configuring, and tuning traditional on-premises data engineering toolchains.
• Adaptability within a highly fluid, constantly evolving start-up paradigm, high ownership habits, and clear communication metrics to align with distributed technical and business squads.